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Second generation DSP software for picture rate conversion 

R.B. Wittebrood G. de Haan 



Introduction 
Advances in multimedia have led to^ a large range, 
of video formats and, more specific, an increasing 
number of picture update frequencies. Therefor, mis- 
matches between the, PC or TV, display rate and 
the video rate increasingly occur and conversions are 
necessary for interfacing. The complexity of the ap- 
plied algorithms range from very simple, e.g. picture 
repetition, to highly advanced applying motion esti- 
mation and compensation techniques. The more so- 
phisticated methods are necessary to prevent motion 
judder and motion blur. Dedicated consumer ICs for 
high quality picture rate conversion have been avail- 
able for some years. Recently, algorithmic progress 
resulted in a first generation high quality picture rate 
conversion in software that runs real time on a DSP 
[1, 2]. Since then a number of significant algorith- 
mic advances caused a considerably improved perfor- 
mance. The most relevant progress was made in the 
algorithm for motion estimation (ME). This paper 
presents the new ME algorithm, while we refer to [1] 
for the motion compensation and de-interlacing. 

The new motion estimator 
The ME is a further development of the concept de- 
scribed in [1], which aims at estimating motion on 
an object basis and uses sub-sampled images (4 x 
hor., 2 x ver.). Compared jto [1], the parameters of 
the object motion models, P Q (n) = (s x , s yi d Xl dy) are 
extended with two additional coefficients^ d x and dy. 
As in [1], the parameter vector of an object is de- 
termined on a small set of "interesting" image parts, 
Sly focusing objects by giving different weights to 
different positions. High weights (> 1) are given to 
positions where the model resulted to be best in the 
previous segmentation, and low weights (< 1) are 
given to areas where competing models turned out 
to be better. The actual object parameter vector is 
a result of a log-search through the parameter vector 
space, optimising the error criterion that remained 
unchanged: 

where F s (x,n) is the luminance value at position x 



of the sub-sampled image with index n, C 0 (x,n) is 
the vector resulting from candidate model C 0 (n) at 
"position^ and W 0 (S) is the weight at position x 
An important improvement occurred in the segmen- 
tation. Previously ([1]), the segmentation mask as- 
signed the candidate object o with the lowest local 
modified cost function e 0 (X,n) to the block B(X): 

£o(*,n) = 

- F.(S-5 0 {S,n),n-l)\ + P(X,n) 

A first modification is that we allowed a small devia- 
tion of the local motion vector from the object motion 
model, using a bias vector B(X>n); 



D 0 (X 



,n)=( 



s x (o,n) + Xd x 
s v (o,n) +Yd y 



s (o,n) \ 
(o,n) J 



+ B(X,n) 



(3) 

D 0 (Xj n) is the motion vector determined by object o 
and the bias field B(X, n). This allows more complex 
object motions to be modelled than with the 4 simple 
objects alone. 

Another difference is that we added a penalty, 
P(X,n), to the error function in eq. (2), while the 
evaluated object vectors are reduced to a subset of the 
available object models, using values occurring in a 
spatio-temporal neighbourhood, adding their corre- 
sponding bias vectors and extending this set with a 
random candidate for the bias vector and a single ob- 
ject vector that does not occur in the neighbourhood. 
The penalties are selected smaller for the "neighbour- 
ing" candidates and higher for the randomly selected 
vectors, similar to what has been described in [3]. 
The effect of this modification is an increased spatio- 
temporal consistency of the vector field and a reduced 
operations count, as only a limited number of incon- 
sistent vectors are evaluated. 

The set of "interesting" image parts, or points of in- 
terest (POIs), is determined by selecting a fixed num- 
ber of samples from the set of blocks for which the 
error criterion, eq. (1) is above a threshold Tpoi for 
C?o(x,n) = 0. In figure 1 an example is given of the 
POIs and the segmentation mask. Figure 2 gives a 
block diagram of the new ME. 
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A refinement bias field improving an object motion estimator 



G. de Haan and R.J. Wittebrood 



Philips Research Laboratories, Television Systems Group, 
Prof. Holstlaan 4, 5656 AA Eindhoven, The Netherlands 



A "bias" field for an earlier designed 
object based motion estimator is proposed, suit- 
able for high-quality motion compensated scan-rate 
conversions. Compared to the prior art, the advan- 
tage of the resulting object motion estimator with 
bias field is that the object motion can be mod- 
elled with relatively simple motion models, while 
achieving high accuracy, as the bias fied describes 
the (small) deviations from the simple motion mod- 
els used. 

Keywords: motion estimation, parametric motion 
models, image segmentation, scan rate conversion. 

1 Introduction 



of the image should be able to compete in operations 
count with a block based motion estimator, as one 
should expect less objects than blocks in realistic im- 
ages. It is only in the assignment of image parts to 
objects that an effort is required comparable to the 
evaluation of candidate vectors on block basis. If the 
number of objects does not exceed the number of candi- 
date vectors too much, the overhead of an object based 
motion estimator should be neglegible. It is assumed 
here that the motion per object can be described with 
fairly simple parametric models (, corrected with a local 
bias vector). 

In the following subsections, we shall describe the mo- 
tion model used, the estimation of the model param- 
eters, the cost function used, and the new recursive 
segmentation of the image. 



The motion estimator described in this paper was de- 
signed to be suitable for scan rate conversion, with a 
computational complexity suitable for consumer elec- 
tronics application, i.e. comparable to [5, 6]. 

The most striking characteristic of the object motion 
estimator described earlier in [1], is that no effort is 
put in segmenting the image into objects prior to esti- 
mation of the model parameters. Basically, a relatively 
small number of interesting image parts is selected, and 
a number of parallel motion model parameter estima- 
tors is trying to optimize their parameters on this data 
set. As soon as one of the estimators is more successful 
than another in a certain number of interesting image 
parts, it is focused on those parts, whereas the remain- 
ing estimators focus on the other parts. In short: in- 
dividual estimators try to conquer image parts from 
eachother, dividing the total image into "objects". 

Fundamentally, such an object-based motion estima- 
tor 1 that wastes no effort in expensive segmentation 

1 An object* in the context of this estimator, refers to image 
parts that can be described with the same motion model. These 
image parts do not necessarily correspond to a single physical 
object in the scene. 



1.1 The motion model 

To keep complexity low, the motion of each object o is 
described by a simple first order linear model that can 
only describe translation and scaling 2 , 

J3.(*,n)=( ^n)+xd x (o,n) \ 
\ s y {o,n) + ydy(o,n) ) 

using 5 0 (x,n) for the displacement vector of object o 
at location x = ( x ] in the image with index n. 



1.2 Parameter estimation 

Given a motion model, next its parameters need to be 
optimised for a given object in the image. As station- 
ary image parts occur in almost every sequence, we 

3 More complex parametric motion models have been pro- 
posed and can indeed be applied in combination with the pro- 
posed algorithm, but will be disregarded here, as we shall intro- 
duce a refinement that makes such complex models obsolete. 
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is required on the sub-sampling grid. 




Figure 2: Example of selection of points of interest. The 
grey blocks are emphasized pixel blocks of interest in PEi , 
the white ones are emphasized in PE2- 

o The pixel values are multiplied with a second 
weight factor smaller than 1, e.g. 0.1, in case the 
segmentation assigned the position to another pa- 
rameter estimator and this estimator achieved low 
match errors. 

Figure 2 gives an example of the selection of the pixel 
blocks of interest in an image with a single moving 
object and a moving background. The moving back- 
ground of the image is object o — 1, and the bicyclist 
is object o = 2. Both parameter estimators are opti- 
mized on the same set containing the blocks of interest, 
but as soon as one estimator is selected in the segmen- 
tation to be best in an area, the pixel block of interest 
in that area is emphasized in the cost function. Af- 
ter a while, this converges to the situation illustrated, 
where one estimator fo cusses om the grey blocks and 
the other on the white pixel blocks in 5/(n). 

More formally, the cost function is calculated according 
to: 

e(<?o,n) = 52 W 0 (xy\F s (x,n)-Fs(x-C 0 (x,n),n-l)\ 

x&IS g ^ 

where F s (x,n) is the luminance value at position x 
in a sub-sampled image with index n, and C 0 (x,n) is 
the vector resulting from candidate model C 0 (n) at 
position x. 

The sub-sampling effectively reduces the required 
memory bandwidth. Images are sub-sampled with a 
factor of four horizontally and a factor of two vertically 
on a field base, generating a sub-sampled image F a (n) 
from each original field F(n). In order to achieve pixel 
accuracy on the original pixel grid of F, interpolation 



Figure 3: Picture and the resulting segmentation. 



1.4 Recursive segmentation 

The segmentation is the most critical step in the algo- 
rithm. Its task is to assign one motion model to each 
group of pixels. This is basically achieved by assign- 
ing the best matching model to each group of pixels (a 
block B, which is typically as large as 8 x 8 pixels on 
frame base). 

For each block, a block match error, e 0 (-Y,n) corre- 
sponding to each of the estimated parameter vectors. 
P 0 , can be calculated according to: 

€ 0 (X,n)= ^ |F,(x + (l-a)A>(x f n),n)- 

F s (x- aA,(x,n),n-l)| 

(6) 

The temporal instance where this segmentation is valid 
is defined by a. 

The straightforward equivalent of the full-search block 
matcher, would result if the segmentation mask 
M(X,n) assigned the object o with the lowest block 
match eror, e 0i to the block B(X). Such a segmen- 
tation is more dangerous when many objects occur in 
the image, since inconsistencies, similar to those of a 
full search block matcher, are not prevented. 

Therefore, we adopted a recursive segmentation 
method that more closely resembles the strategy of 
the 3-D RS block matcher, i.e. use spatial and tem- 
poral predictions of the best PE 0 {n) and penalise 
chosing a PE 0 {n) that does not occur in the spatio- 
temporal neighbourhood. Formally, the segmentation 
mask M(X t n) assigns the object o with the lowest lo- 
cal modified cost function e 0 '(X,n) to the block B(X), 
where 

U =e„ + P{X,n) (7) 
while P(X,n) is a penalty choosen according to the 
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Figure 1: Block diagram of multiple parameter estimators 
and segmentation. 



no estimation effort is required to make this available. 
The parameter vectors of additional objects, o, o > 0, 
are estimated separately, in parallel, by their respec- 
tive parameter estimators (PE Q ), as shown in Figure 
1. Each PE 0 updates a previously estimated param- 
eter vector, after which the best parameter candidate 
vector, according to a cost function, is selected as the 
result parameter vector for that object. 

Considering the four parameter model of eq. (1), the 
parameters of object o, o > 0. are regarded as a pa- 
rameter vector P Q : 



Po{n) = 



/ s x (o,n) \ 
5 y (o, n) 
d x {o, n) 
\ d y (o,n) J 



(2) 



and we define our task as to select P 0 (n) from a number 
of candidate parameter vectors C 0 (ri) as the one that 
has the rninimal value of a cost function, to which we 
shall return lateron. 

The candidates are generated much similar to the 
strategy exploited in [5. 6], i.e. take a prediction vec- 
tor, add at least one update vector, and select the best 
candidate parameter vector according to an error crite- 
rion. Candidate parameter set CS 0 (n) contains three 



candidates C 0 (n) according to: 

CS 0 {n) = {C 0 (n)\C o (n) = P G (n - 1) + mU 0 (n). 

U Q {n) € US 0 (n). 

m = -1,0.1} 

(3) 

with update parameter U 0 (n) selected from update pa- 
rameter set~USo(n)i 

/: \ /0 \ / 0\ / 0\ " 
i 
0 



US 0 (n) = 



0 
0 



\ o / 



0 



0 
0 

V i J 



(i = l,2,4,8,16) 



(4) 



1.3 The cost function 



Given the motion model and some candidate parame- 
ter sets, we need to select the best candidate, according 
to a cost function, as the result for a given object. The 
cost function can be a sum of absolute differences be- 
tween motion compensated pixels from neighbouring 
images, with vectors generated with the (Candidate) 
motion model. However, we need to know the area to 
which the motion model is to be assigned. The two 
issues, segmentation and motion estimation, are inter- 
dependent. In order to correctly estimate the motion 
in one object, the object should be known and vice 
versa. 

As a first step in the motion estimation process we 
define a set with pixel blocks of interest. These form 
the set, SJ(n) of "interesting" image parts that will 
be used as a basis for optimization of all parametric 
models. 

Now the focus of the individual parameter estimators 
has to be on different objects. To this end, each pa- 
rameter estimator PE 0 will calculate its cost function 
on the same set of interesting locations defined in set 
51, giving different locations a different weight factor. 
W 0 (X ). The proposed algorithm is straightforward: 

o The pixel values are multiplied with a first weight 
factor larger than 1, e.g. 8, in case the pixel in 
SI{n) belonged to the same object, i.e. the same 
parameter estimator, according to the previous 
image segmentation step. 

o The pixel values are multiplied with a second 
weight factor smaller than 1, e.g. 0.1, in case the 
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following rule: 



(M(X+6,n) = o) 
(M(X -5,n- 1) = o) 
(otherwise) 



and 



6= (j.),i,i = 0,±l 



(8) 



(9) 



Similar to what has been suggested for the 3-D RS 
block matcher, P u is the largest penalty, P t just a small 
one. while there is no reason why P s could not just 
be zero. A fairly obvious simplification is to fix 8 to 
the direction opposite to the scanning direction, and 
to alternate the scanning from field to field. Figure 3 
gives an example of a segmentation according to the 
object-based motion estimation method, with the orig- 
inal luminance image. 
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Figure 1: Block diagram of multiple parameter 
and segmentation. 



estimators 



using D 0 (x,n) for the displacement vector of object o 
at location x = in the image with index n. 



Po(n) = 



s x (o y n) \ 
s y (o,n) 
d z (o, n) 
d v (p,n) J 



(2) 



and we define our task as to select P 0 (n) from a number 
of candidate parameter vectors C 0 (n) as the one that 
has the minimal value of a cost function, to which we 
shall return lateron. 



The candidates are generated much similar no rhe 
strategy exploited in [?, ?], i.e. take a prediction vec- 
tor, add at least one update vector, and select the best 
candidate parameter vector (eq. ??). Candidate pa- 
rameter set CS 0 (n) contains three candidates C 0 {n) 
according to: 

CS 0 (n) = {C Q (n)\Co(n) = P Q {n - 1) + mU»(n)\ 

U 0 (n) £ L'So(n). 
rn = -1.0. 1} 

with update parameter 0 o {n) selected from update pa- 
rameter set US 0 {n): 



D 0 (x.n)- ^ $y{0 ,n)+yd y (o 7 n) ) (1) 0 S 0 (n) = < 



if* \ 



0 
0 



( 0 

i 
0 

V o ) 



f o \ 

0 

i 

V° J 



( o \ 

0 
0 

V i / J 



1 



(i = 1.2.4.3.16) 



(41 



1.2 Parameter estimation 

Given a motion model, next its parameters need to be 
optimised for a given object in the image. As station- 
ary image parts occur in almost every sequence, we 
assume the presence of an 'object o, o = 0\ for which 
motion is described by, 0. the zero vector. Clearly 
no estimation effort is required to make this available. 
The parameter vectors of additional objects, o, o > 0. 
are estimated separately, in parallel, by their respec- 
tive parameter estimators (PE 0 ), as shown in Figure 
1. Each PE 0 updates a previously estimated param- 
eter vector, after which the best parameter candidate 
vector, according to a cost function, is selected as the 
result parameter vector for that object. 

Considering the four parameter model of eq. (1). the 
parameters of object o, o > 0, are regarded as a pa- 
rameter vector P 0 : 



1.3 The cost function 

Given the motion model and some candidate parame- 
ter sets, we need to select the best candidate, according 
to a cost function, as the result for a given object. The 
cost function can be a sum of absolute differences be- 
tween motion compensated pixels from neighbouring 
images, with vectors generated with che (Candidate) 
motion model. However, we need to know tIlc area ro 
which the motion model is to be assigned. The two 
issues, segmentation and motion estimation, are inter- 
dependent. In order to correctly estimate the motion 
in one object, the object should be known and vice 
versa. 

As a first step in the motion estimation process wo 
define a set with pixel blocks of interest, or feature 
points. These form the set. SF(n) of ''features'' that 
will be used as a basis for optimization of ail parametric 
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the image, since inconsistencies, similar to those of a 
full search block matcher, are not prevented. 

Therefore, we adopted a segmentation method that 
more closely resembles the strategy of the 3-D RS block 
matcher, i.e. use spatial and temporal predictions of 
the best PE 0 (n) and penalise chosing a PE 0 {n) that 
does not occur in the spatio-temporal neighbourhood. 
Formally, the segmentation mask M (X.n) assigns the 
object o with the lowest local modified cost function 
e 0 ' (X , n) to Jthe block B{X_ ) , where 



e 0 =£ 0 +P(X,n) 



IT) 



while P(X z n) is a penalty choosen according to the 
following rule: 



P(X,n) = 




,(M(.Y + £n) =o) 
,(M(-Y-£n-l) = o) 
, (otherwise) 



and 



(5= Q^i : i = 0,±l 



(8) 



(9) 



Similar to what has been suggested for the 3-D RS 
block matcher, P u is the largest penalty. P t just a small 
one, while there is no reason why P s could not just 
be zero. A fairly obvious simplification is to fix 5 to 
the direction opposite to the scanning direction, and 
to alternate the scanning from field to field. Figure 3 
gives an example of a segmentation according to the 
object-based motion estimation method, with the orig- 
inal luminance image. 



2 Selection of the feature points 

As a first step in the motion estimation process we 
defined a set with "features" (see section 1.3). This 
set, 5F(n) of " interesting 1 ' image parts is used as a 
basis for optimization of all parametric models. 

An advantegeous way to obtain this set, to be used in 
the parameter estimation of the next picture pair, is to 
fill it with all blocks (on the sub-sampled image grid) 
that don't match well (threshold) with the correspond- 
ing blocks in the previous image: 



SF(n) = {.Y|€ 0 (-Y, n - 1) > Th) 



(10) 



where e Q = e 0 for o = 0. 

This set, because of the threshold, contains no flat ar- 
eas, as these would not help in the minimization of 



the cost function and therefore unnecessarily increase 
the operation count. Furthermore, the algorithm elim- 
inates all stationary image parts from feature set, even 
if they contain high contrast details. That is another 
reduction of the number of irrelevant image parts, since 
the parameter^est imator" gene rating the ''zero veloc- 
ity model" requires no optimization on pixel data. In 
other words: the set of features contains blocks with 
significant detail in non-stationary image parts, exactly 
the best basis for all parameter estimators for non- 
stationary- objects 

The threshold can be determined with a control loop 
that keeps the size of the set more or less constant. As 
an alternative, a relatively small fixed threshold can be 
used, such that always more than enough image parts 
are contained in an initial set. In this case, the final 
set results by sub-sampling the initial set with a factor 
such that a more or less constant number of image 
parts results. 
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A "meta model" is proposed for an ear- 
lier designed object based motion estimator, suit- 
able for high-quality motion compensated scan- 
rate conversions. The meta model is a parametric 
model, with a larger number of parameters than 
the models used to describe the objects in the ob- 
ject motion estimator, with its parameters derived 
'from the output vector field of this object motion 
estimator, and acting as a candidate object in the 
segmentation stage of the object motion estimator. 
Compared to the prior art, the advantage of the 
resulting object motion estimator with this meta 
model is that an accurate model of complex global 
motion can be added as a powerful candidate to im- 
prove the output vector field, with a relative small 
calculational effort. 

Keywords: motion estimation, parametric motion 
models, image segmentation, scan rate conversion. 



1 Introduction 

The motion estimator described in this paper was de- 
signed to be suitable for scan rate conversion, with a 
"computational complexity suitable for consumer elec- 
tronics application, i.e. comparable to [6, 7). 

The most striking characteristic of the object motion 
estimator described earlier in [1], is that no effort is 
put in segmenting the image into objects prior to esti- 
mation of the model parameters. Basically, a relatively 
small number of interesting image parts is selected, and 
a number of parallel motion model parameter estima- 
tors is trying to optimize their parameters on this data 
set. As soon as one of the estimators is more successful 
than another in a certain number of interesting image 
parts, it is focused on those parts, whereas the remain- 
ing estimators focus on the other parts. In short: in- 
dividual estimators try to conquer image parts from 
eachother. dividing the total image into "objects". 



Fundamentally, such an object-based motion estima- 
tor 1 that wastes no effort in expensive segmentation 
of the image should be able to compete in operations 
count with a block based motion estimator, as one 
should expect less objects than blocks in realistic im- 
ages. It is only in the assignment of image parts to 
objects that an effort is required comparable to the 
evaluation of candidate vectors on block basis. If the 
number of objects does not exceed the number of candi- 
date vectors too much, the overhead of an object based 
motion estimator should be neglegible. It is assumed 
here that the motion per object can be described with 
fairly simple parametric models(, corrected with a lo- 
cal bias vector, according to an earlier invention dis- 
closure). 

In this disclosure, a "meta model" is proposed to im- 
prove this object motion estimator. The meta model is 
a parametric model, with a larger number of parame- 
ters than the models used to describe the objects in the 
object motion estimator, with its parameters derived 
from the output vector field of this object motion esti- 
mator, and acting as a candidate object in the segmen- 
tation stage of the object motion estimator. Compared 
to the prior arx. the advantage of the resulting object 
motion estimator with this meta model is that an ac- 
curate model of complex global motion can be added 
as a powerful candidate to improve the output vector- 
field, with a relative small calculational effort. The 
proposed improvement is related to the improvement 
of the 3-D RS block matcher with a candidate vector 
derived from a parametric motion model, as has been 
documented in [3. 4j. 

In the following subsections, we shall describe the mo- 
tion model used in the prior art object motion estima- 
tor, the estimation of the model parameters, the cost 
function used, the recursive segmentation of the im- 



l An object, in the context of this estimator, refers to image 
parts that can be described with the same motion model. These 
image parts do not necessarily correspond to a single physical 
object in the scene. 
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Figure 2: Example of selection of points of interest. The 
grey blocks are emphasized pixel blocks of interest in PEi, 
the white ones are emphasized in PE2. 



to a cost function, as the result for a given object. The 
cost function can be a sum of absolute differences be- 
tween motion compensated pixels from neighbouring 
images, with vectors generated with the (Candidate) 
motion model. However, we need to know the area to 
which the motion model is to be assigned. The two 
issues, segmentation and motion estimation, are inter- 
dependent. In order to correctly estimate the motion 
in one object, the object should be known and vice 
versa. 

As a first step in the motion estimation process we 
define a set with pixel blocks of interest. These form 
the set, SI(n) of " interesting" image parts that will 
be used as a basis for optimization of all parametric 
models. 

Now the focus of the individual parameter estimators 
has to be on different objects. To this end. each pa- 
rameter estimator PE 0 will calculate its cost function 
on the same set of interesting locations defined in set 
SI, giving different locations a different weight factor. 
H'o(.Y). The proposed algorithm is straightforward: 

o The pixel values are multiplied with a first weight 
factor larger than 1, e.g. 8, in case the pixel in 
SI(n) belonged to the same object, i.e. the same 
parameter estimator, according to the previous 
image segmentation step. 

o The pixel values are multiplied with a second 
weight factor smaller than 1, e.g. 0.1, in case the 
segmentation assigned the position to another pa- 
rameter estimator and this estimator achieved low 
match errors. 



Figure 2 gives an example of the selection of the pixel 
blocks of interest in an image with a single moving 
object and a moving background. The moving back- 
ground of the image is object o = 1, and the bicyclist 
is object o = 2. Both parameter estimators are opti- 
mized on the same set containing the blocks of interest, 
but as soon as one estimator is selected in the segmen- 
tation to be best in an area, the pixel block of interest 
in that area is emphasized in the cost function. Af- 
ter a while, this converges to the situation illustrated, 
wher e one es timator foc usses om the grey blocks and 
the other on the white pixel blocks in SI{n). 

More formally, the cost function is calculated according 
to: 

e(C 0 .n) a Yi W 0 (z)^F s (x t n)-F s (x-C 0 (x,n).n-l)\ 
16/s 

(5) 

where F s (x,n) is the luminance value at position x 
in a sub-sampled image with index n. and C 0 (f. n) is 
the vector resulting from candidate model C 0 {n) at 
position x. 

The sub-sampling effectively reduces the required 
memory bandwidth. Images are sub- sampled wirh a 
factor of four horizontally and a factor of two vertical lv 
on a field base, generating a sub-sampled image F s {n) 
from each original field F(n). In order to achieve pixel 
accuracy on the original pixel grid of F, interpolation 
is required on the sub-sampling grid. 




Figure 3: Picture and the resulting segmentation. 



1.4 Recursive segmentation 

The segmentation is the most critical step in the algo- 
rithm. Its task is to assign one motion model to each 
group of pixels. This is basically achieved by assign- 
ing the best matching model to each group of pixels (a 
block B, which is typically as large as 8 x 8 pixels on 
frame base). 

For each block, a block match error, e 0 (JY.n) corre- 
sponding to each of the estimated parameter vectors. 
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Claims 

Claim 1: A method, and apparatus realizing this 
method, for estimating motion from video data, 
comprizing: - — 



generating 
models 



at least two motion parameter 



o furnishing candidate parameter models for 
every picture part (block); 

o assigning either of the candidate parameter 
models to blocks, using a criterion function: 

o furnishing a bias field, describing local de- 
viations from the selected motion parameter 
model 

Claim 2: A method, and apparatus realizing this 
method, according to claim 1, in which the afore- 
mentioned segmentation is a recursive segmenta- 
tion, i.e. selects the motion parameter model for 
the current block from a set containing motion 
models selected in a spatio-temporal neighbour- 
hood, plus at least one additional motion model. 

Claim 3: A method, and apparatus realizing this 
method, according to the previous claims, in 
which the bias field is generated according to the 
following steps: 

o fetch at least one prediction bias value from 
a spatio-temporal neighbourhood; 

o add a bias update value to at least one of the 
bias prediction values; 

o assign the best bias value, according to a cri- 
terion function, to the current block 

Claim 4: A method, and apparatus realizing this 
method, according to any of the previous claims, 
in which the criterion function mentioned in claim 
3 is a match error criterion, e.g. a sum of abso- 
lute pixel differences, calculated between a block 
in the current picture and a block in the previ- 
ous picture shifted over a motion vector resulting 
from the motion model selected for the current 
block and the candidate bias field value. 



Claim 5 A method, and apparatus realizing this 
method, for segmenting pictures into areas (ob- 
jects) the motion of which is described described 

- by a single motion parameter- model, comprising 
the steps of: 

o furnishing (, for a group of pixels) of at least 
one (segmentation) prediction, taken from a 
spatio- temporal neighbourhood of the seg- 
mentation mask; 

o selecting (, for that group of pixels) at least 
one additional segmentation candidate; 

o deciding upon the segmentation (, for that 
group of pixels) using a criterion function. 

Claim 6 A method, and apparatus realizing this 
method, according to claim ^ in which the afore- 
mentioned criterion function is a match error, e.g. 
a sum of absolute pixel differences. 

Claim ^ A method, and apparatus realizing this 
method, according to the previous claims 5^6^ 
Mi which the match error for additional segmentation 
candidate is increased with a "penalty"' value. 

Claim 3 A method, and apparatus realizing this 
method, according to any of the previous claims, 
in which the processing order of the picture parts 
(blocks, or groups of pixels) is different for succes- 
sive iterations, or successive fields. 



